Computer vision-based methods for enhanced jbig2 and generic bitonal compression

ABSTRACT

A system and method of symbol matching may include a processor configured to determine which pixels of a first symbol are tangent pixels; for each of the determined tangent pixels, determine whether a second symbol includes a pixel corresponding to the tangent pixel that includes at least one same tangent constraint as that of the tangent pixel; accept the first and second symbols as a match based on shared tangent constraints conditional upon a determination that the second symbol includes for each of at least a subset of the tangent pixels of the first symbol a corresponding pixel that includes the at least one same tangent constraint as that of the tangent pixel; and generate a document including a single symbol that is mapped to the first and second symbols.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/198,055, filed Aug. 25, 2008, which claims priority to U.S.Provisional Patent Application No. 60/966,094, filed Aug. 24, 2007,entitled “Computer Vision-Based Methods For Enhanced JBIG2 And GenericBitonal Compression,” the contents of each of which are hereinincorporated by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates to a system and method of bitonal imagecompression using human perception modeling.

BACKGROUND INFORMATION

Conventional data compression methods analyze pixels of a file todetermine which glyphs composed of sets of the turned on pixels (e.g.,black pixels) are matches. They then proceed to associate the matchedglyphs (alternatively referred to as nodes, connected components, cc's,or bitmaps) with a common model. When comparing two matching functions(also referred to herein as “matchers”), one can be described as atighter matcher if it is stricter with respect to which fonts it allowsto be matched, while the other can be described as looser if it is morelenient in this regard. For example, a Hausdorff matcher requires thatevery black pixel (e.g., position {x, y}) on a first bitmap must find acorresponding black pixel on a second bitmap to which the first iscompared within a pixel distance of one. A Rank 95 Hausdorff requiresthat 95% of the black pixels find a corresponding black pixel within adistance of one. A Rank 95 Hausdorff matcher is therefore looser than aHausdorff matcher. A Quadrant Hausdorff matcher requires that everyblack pixel finds a corresponding black pixel within distance one in thesame quadrant direction. (See U.S. Pat. No. 6,748,115 FIGS. 9 & 10 foran illustration.) Every black pixel in the first bitmap which does notfind a matching pixel in the same exact position of the correspondingbitmap, must find a matching pixel in the same quadrant. For example, ifall such pixels find a match in the top right quadrant, the bitmapsmatch. However, if there exists a pixel which needs to find support in adifferent quadrant such as the bottom left, the match would not beallowed. A Quadrant Hausdorff matcher is tighter than a Hausdorffmatcher, as it imposes an additional directional constraint. Applying alooser matcher may result in fewer models overall, but increase thelikelihood of a mismatch; applying a tighter matcher may result in morefont models overall, but decrease the likelihood of a mismatch.

Most JBIG2 implementations will generally use a range of matchers,depending on the properties of the bitmaps being compared. Theseproperties may include the height, width, area, the number of holes inthe bitmap, and mean stroke thickness of the bitmaps. If the bitmaps arelarge and have a wide mean stroke thickness, they can generally besafely matched with a loose matcher such as a Rank 95 Hausdorff. If thebitmaps are smaller and thinner, they will be more likely to need atighter matcher such as a Quadrant Hausdorff.

However, a general tradeoff in the field of data compression is rate vs.distortion, i.e., the higher the compression rate, the greater theamount of distortion. In fact, the tradeoff, referred to herein as “ratedistortion theory,” is a major branch of information theory, the problemof determining the minimal amount of entropy (information) R that shouldbe communicated over a channel, so that the source (input signal) can beapproximately reconstructed at the receiver (output signal) withoutexceeding a given distortion D.

Rate distortion theory, created by Claude Shannon in his foundationalwork on information theory, gives theoretical bounds for how muchcompression can be achieved using lossy data compression methods. Manyof the existing audio, speech, image, and video compression techniqueshave transforms, quantization, and bit-rate allocation procedures thatcapitalize on the general curve of the rate-distortion functions.

In rate distortion theory, the rate is the number of bits per datasample to be stored or transmitted. The notion of distortion is asubject of on-going discussion. In the most simple case (which isactually used in most cases), the distortion is defined as the varianceor the means squared error of the difference between input and outputsignal. However, since most lossy compression techniques operate ondata, e.g., music, pictures, video, that will be perceived by humans,the distortion measure preferably should include some aspects of humanperception. Audio compression perceptual models, and thereforeperceptual distortion measures, are relatively well developed androutinely used in compression techniques such as MP3, but are often noteasy to include in rate distortion theory, i.e., calculation of thedegree of distortion is difficult when perception models are used. Inimage and video compression, the human perception models are less welldeveloped and inclusion is mostly limited to the JPEG and MPEG weighting(quantization) matrices.

Shannon's rate distortion theory notwithstanding, there does not seem tobe an inherent tradeoff between rate and distortion. For example, in thelossless data compression domain, studies have shown that human entropyfor English language text is about 1 bit per character (bpc), (i.e., aprobability of a human to correctly guess a next character correspondsto a probability for which only 1 bit per character would be requiredfor the encoding.) Traditional lossless text compression based onLempel-Ziv methods (e.g., zip) has a compression rate of about 2 bpc.Newer lossless text compression methods that utilize techniquesincluding arithmetic encoding and Markov models (of order statistics),such as PPMD, achieve a compression rate of approximately 1.5 bpc, whichis closer to, but still higher than, human entropy rates of roughly 1.1bpc. These improvements in lossless text compression rates, achievedover a 20 year period, did not come at a cost of greater data distortionsince both zip and prediction-by-partial-matching (ppm) methods areentirely lossless. To the contrary, PPMD is a better model for text thanthe traditional lossless text compression methods, and better modelingyields a better representation or understanding of the data, andconsequently, achieves lower entropy.

Modeling human perception is a very effective tool for efficient datacompression methods. Although modeling human perception has been veryimportant in areas like computer vision for a long time, it hasgenerally played a very tangential role in data compression. Humanperception models have evolved over time and seem to be highlyeffective. Rather than perceive things at a sensor level, e.g., pixels,perception is done at an object level. This has the positive effect ofseparating out signal from noise, as well as greatly reducing the amountof information that needs to be retained.

For example, a color image scan at 300 dots per inch (dpi) mighttypically involve 300*300 (pixels per square inch)*8.5*11 (paper size ininches)*24 (bits per pixels) equal to approximately 202 million bits ofinformation (202 Mb). A standard compression method would involvefinding ways to save this information, i.e., 202 million pieces ofinformation, with a minimal file size and minimal distortion (variance)between the input and output signals. Nowhere in typical imagecompression algorithms is “scene understanding” an essential, or evenimportant, component. Human perception works very differently: firstthere is an understanding of the general scene (e.g., indoor, outdoor,document, invoice document, etc.). Once the scene is understood on somebasic level (e.g., frame instantiation as defined by Patrick Winston),the image is grouped or segmented into objects, of which there are veryfew compared to the number of pixels and corresponding bits. Forexample, if a color image has over 200 million bits of information, ahuman may typically perceive many fewer objects, perhaps less than onehundred. For a bitonal image scan, a typical compression algorithm(e.g., CCITT4) views the problem as storing 8 million pieces of data(pixels or bits), while a system using human perception models, referredto herein as a human system, perceives the problem as understanding500-2000 character symbols or connected components.

Perceptually lossless image compression is about image understanding.The fundamental tenet in perceptually lossless image compression is tomodel human perception as closely as possible. Human perception modelsfor speech, image, and video are typically much more advanced thanstandard computer models used in compression and other fields that donot attempt to model human perception. As such, perceptually losslesscompression can achieve much lower rates of compression, ideally with NOperceptual distortion. On the other hand, a disadvantage of suchperceptual methods is that they are domain specific, so that differenttechniques are used, for example, in image compression, speechcompression, and video compression.

SUMMARY

Embodiments of the present invention provide a system and method forcompressing bimodal image documents in a manner that approximates humanvisual methods. With respect to JBIG2 compression, a matcher mayrepresent a document image as a set of components, rather than as a setof pixels. This may typically reduce the number of objects from 8million pixels to fewer than 3,000 components per page. These componentsthemselves may generally be digitized elements within some font library,rather than random components. The system and method may match similarcomponents together, just as the human visual system, i.e., theperception of the human mind, would at some point group together similarfont components, even if the base language were otherwise unknown, andkeep track of them internally by referencing the same datarepresentation. The system and method of the present invention may grouptogether sequences of models to form composites, where a composite modelin JBIG2 is a series of individual models that follow one another in aseries.

JBIG2 encoding may use pattern matching and substitution methods, wheremultiple instances of the same font in the original image are all placedinto the same class and associated with a single model and areaccordingly represented by the same bitmap in the encoded image. Thesystem and method of the present invention may implement a matchingfunction to determine if two fonts are the same, where different typesof matching functions are used depending on the situation. Failure tomatch two fonts which can be safely merged may result in additionalmodels, which may increase file size. On the other hand, mistakenlymatching two fonts which should be kept separate can result in visibledistortion in the encoded image. The system and method of the presentinvention may approximate human performance, merging all nodes that areinstantiations of the same symbolic font model as determined on acomponent level rather than a pixel level, without ever merging twodistinct node classes together for association with a single model whenthey are not instantiations of the same symbolic font.

In an example embodiment of the present invention, the compressionsystem and method may group symbolic sequences together, for example,just like the human visual system is adept with grouping togethersymbolic strings and storing them as a single entity, such as words,which may afford greater compression and may lead to word-based JBIG2coding. The system and method of the present invention may includecontext-sensitive matching, akin to human learning. In this framework,given a particular set of matching functions, the system and method ofthe present invention may match two glyphs (alternatively referred to asnodes, connected components or cc's) that might otherwise not beconsidered a match when they are not provided in the same context. Twoglyphs may have the same context when one or more of their respectiveneighbors belong to the same model class.

Example embodiments of the present invention provide enhancements to theart of bitonal image compression in several different areas. These areasinclude improved shape matching and segmentation methods.

Example embodiments of the present invention provide improved shapematching methods, consistent with human perception, that result invirtually no model fragmentation (redundant models) or mismatches. Inaddition to new matchers, example embodiments of the present inventionprovide new methods for determining when a looser matcher can be safelyused. Example embodiments of the present invention also provide fast(approximate) methods to construct graphs, image grids, and other datastructures that are very useful for computing shape similarity, asignificant aspect of efficient JBIG2 compression. Example embodimentsof the present invention provide improved techniques for efficientlossless bitmap encoding using these shape similarity data structures,e.g., a shape-based approximate min spanning forest.

Segmentation is a significant aspect of efficient compression, bothbitonal and color. Example embodiments of the present invention providenumerous segmentation enhancements, including picture extraction andtext region grouping, all akin to human perceptual segmentationprocesses, and other bitonal compression enhancements, such as improvedJBIG2 addressing using vertical padding and improved image processingmethods for grouping together connected components.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates a data compressor and dataencoder, according to an example embodiment of the present invention.

FIGS. 2 to 7 show pairs of bitmaps that include minor variations whichmay be treated significantly by a tangent matcher, according to anexample embodiment of the present invention.

FIG. 8 shows a bitmap including an on pixel having a tangent supportused as a criterion by a neighbor-supported tangent matcher, accordingto an example embodiment of the present invention.

FIGS. 9 and 10 show a pair of bitmaps having structures with differentdegrees of sharpness, which may be used by feature-based matcher toapply different matching criteria, according to an example embodiment ofthe present invention.

FIGS. 11 and 12 show bitmaps treated differently based on differenceswith respect to symmetry, according to an example embodiment of thepresent invention.

FIG. 13 shows a bitmap with additional white pixels which may beanalyzed by a symmetry matcher, according to an example embodiment ofthe present invention.

FIGS. 14 and 15 show bitmaps sharing local tangent directions but forwhich a match determination may be rejected by a curvature-basedmatcher, according to an example embodiment of the present invention.

DETAILED DESCRIPTION

FIG. 1 shows an example data compressor 102 and data encoder 104 whichmay use analyzer (and/or generator) components 106 to generate acompressed file 110 which represents an input file 100. In an exampleembodiment of the present invention, the data compressor 102 may receivethe input file 100, e.g., a file generated based on a scanned image andincluding glyphs. The data compressor 102 may analyze the glyphs togenerate models, e.g., using some of the analyzer components 106. Anumber of similar glyphs or nodes may form a class associated with asingle model. The data encoder 104 may generate a compressedrepresentation of the file, a compressed file 110, based on the modelsgenerated by the data encoder 104 and some of the analyzer components106. Models and addresses indicating positions for placement of themodels on a page may be used as the compressed file 110. The compressedfile 110 may be decompressed by placing the models in the indicatedpositions, thereby substantially regenerating the input file 100.

Determining Shape Similarity

In an example embodiment of the present invention, the system and methodmay increase efficiency by determining shape similarities of nodes.Based on shape similarity determinations, the system and method maydecide whether to further compare two nodes. Accordingly, the shapesimilarity determinations may decrease the number of nodes to which anode is to be compared, e.g., using a matcher or other comparison.

The system and method may generate second order moments for each node inboth the ‘x’ axis and the ‘y’ axis. In this regard, the system andmethod may determine a node's centroid, which is the position of theaverage position of all on pixels (e.g., black pixels) of the node, andwhich is considered its 0,0 position. The system and method maynormalize all of the node's pixels relative to the centroid position.For example, if a non-centroid pixel's position along the ‘x’ axis is 24and the centroid pixel's position along the ‘x’ axis is 16, then thenormalized position of the non-centroid pixel along the ‘x’ axis is24−16=8. For an axis, the system and method may generate the secondorder moment by obtaining the result of the equation

$\frac{\sum\limits_{x = 1}^{n}x^{2}}{n},$

where x is the node's normalized position along the considered axis, andn is the number of the node's on pixels along the considered axis.

For each axis, after generating the second order moments for all of theconsidered nodes, the system and method may plot the second ordermoments in a histogram, where the x axis of the histogram corresponds tothe generated second order moment values, and the y axis of thehistogram corresponds to the number of nodes having that generatedsecond order moment value. The values of such a histogram may berepresented by the functions H_(X)(z) and H_(y)(z), corresponding to thenumber of nodes with z as the x second moment value, and y second momentvalue, respectively.

For every value z, the system and method may also generate a cumulativehistogram CH_(x)(z) (or CH_(y)(z)). In particular,CH_(x)(z)=[H_(X)(z)+CH_(x)(z−1)], and CH_(y)(z)=[H_(y)(z)+CH_(y)(z−1)].That is, CH_(x)(z) value represents the number of nodes having a secondorder moment value less than or equal to z. For example, if 3 nodes havethe second order moment value 1, 2 nodes have the second order momentvalue 2, and 5 nodes have the second order moment value 3 (all along thex axis), then the H_(x)(1)=3, CH_(x)(1)=3, H_(x)(2)=2, CH_(x)(2)=5,H_(x)(3)=5, and CH_(x)(3)=10.

Using the CH values, the system and method may divide nodes into apredetermined number of bins. For example, if the predetermined numberis 10 and the number of nodes on the page is 5,000, the CH values may beused to divide the nodes into sets of 500 arranged in an array of 10bins, e.g., 500 nodes per bin. Considering both the ‘x’ and ‘y’ axes, atwo dimensional array of bins may be formed. Those nodes having mostsimilar shapes according to the generated second order moments of boththe ‘x’ and ‘y’ axes are grouped together in a single bin. Accordingly,the system and method may determine which nodes have similar shaped to aconsidered node, within a certain range of similarity based on theirproximity to the considered node's bin.

Generating Shape Similarity Space

For solving certain shape similarity problems like K nearest neighborand a shape similarity based MST, it is useful to have pre-computed ashape similarity space. To compute this space, the system and method mayperform the following:

generate a moment profile per component (node or glyph) using 2^(nd)order moments or generate 12 moment profiles per component using three2^(nd) order moments for each of 4 quadrants (3^(rd) order moments canbe used instead of or in addition to the 2^(nd) order moments);

generate a histogram in the ‘x’ and ‘y’ moment directions;

shift the bins' boundaries so that the distribution in each dimension isuniform at k-bins (exemplary preferable settings for k include 32, 16,and 8); and

grid the moment space uniformly to form a shape similarity space, andput each node on its appropriate grid's linked list, based on its momentvalues.

Fast, Approximate K-Nearest Neighbors (AKNN)

AKNN may be used for lossless encoding. Matchers, discussed below, maybe used for lossy encoding.

In an example embodiment of the present invention, after the system andmethod determines that two nodes have shape similarity, the system andmethod may then compare them to make sure that they are to be associatedwith a same model.

For example, for two letters, e.g., b and d, which have the exact samesecond order moments, the system and method may determine that they haveshape similarities and therefore compare them. For the comparison, thesystem and method may overlay the considered two nodes by aligning themby their centroids. The system and method may XOR the aligned nodes. TheXOR value (the number of pixels for which the XOR value is TRUE, i.e.,the pixels at which the aligned nodes differ) may be considered the“distance” between the two nodes.

For each node, the system and method may perform the comparison of thenode to at least a predetermined number of most similarly shaped nodes.If the total of nodes sharing the same bin as the considered node isless than the predetermined number, then the system and method mayspiral out to the bins surrounding the already considered bins until thetotal nodes in all of the considered bins is at least the predeterminednumber. For example, if the predetermined number is 20 and only 5 othernodes share the same bin as the considered node, the system and methodmay compare the considered node to the nodes in the eight binssurrounding the already considered bin. If the total nodes of all nineconsidered bins is less than 20, the system and method may then comparethe considered node to the nodes in the 16 bins surrounding thepreviously considered nine bins, etc. Once the predetermined number ofnodes has been compared to the considered node, the system and methodmay cease spiraling outwards.

After comparing a node to at least the predetermined number of othersdetermined to have similar shapes as described above, the system andmethod may find K closest neighbors to the considered node of the nodesof the considered bins.

In an alternative example embodiment of the present invention, insteadof spiraling outward until at least the predetermined number of nodesare found for comparison with the considered node, the system and methodmay, after each spiral outward, compare the considered nodes to theother nodes of the considered bins and determine whether a predeterminednumber ‘Z’ of nearest nodes have changed. If the nodes belonging to theZ nearest have not changed, the system and method may cease spiralingoutward. Otherwise, the system and method may continue to spiral outwarduntil the nodes belonging to the group of Z nearest neighbors stopschanging.

For example, after comparing the considered node to the nodes sharingthe same bin as the considered node, the system and method may determinewhich Z of the other nodes are nearest to the considered node.Subsequently, the system and method may compare the considered nodes tothose in the eight bins surrounding the considered node's bin. if any ofthe nodes of the eight surrounding bin are determined to be nearer tothe considered node than the already determined Z nearest nodes, theconstitution of the group of Z nearest nodes changes. Accordingly, thesystem and method may spiral outward to compare the considered node tonodes of still more bins.

Thus, for each node in an image, the system and method may findK-Nearest Neighbors by spiraling out, starting at a current gridlocation, until the considered node is compared to N closest nodes ingrid space, e.g., where N=10, 100, 250, or 500, always updating theK-Nearest. When comparing actual shape distance, as an alternative toXOR, Hamming or 12 quadrant moments may be used.

Fast, Approximate Min Spanning Tree (MST) Using Shape Similarity Metric

For some characters, if the character is close enough to anothercharacter, it may be cheaper to encode a difference of the characterfrom the closest other character if the characters are similar enough.Otherwise, it may be cheaper to encode the character from scratch.Conventional spanning trees relate to this principle. Exampleembodiments of the present invention provide efficient ways to generatea spanning tree.

In an example embodiment of the present invention, aside from XORing aconsidered node with the nodes of the surrounding bins as describedabove, the system and method of the present invention may additionallysquare the differences of the second order x of the considered node andthe node to which it is being compared, square the differences of thesecond order y of the considered node and the node to which it is beingcompared, add the squares, and compute the square root of the result(Pythagorean theorem). The result may represent the difference in momentspace between the two nodes. Different numbers of nearest neighbors maybe computed using the two methods, XOR and moment space. For example,the system and method may determine the K nearest neighbors using XORand the M closest using moment space.

After determining the M closest neighbors, the system and method of thepresent invention may build a spanning tree in an efficient manner. Thesystem and method may begin with a first considered node, which thesystem and method may use as a first part of the tree. For each of theconsidered node's M nearest neighbors, the system and method maydetermine whether the neighbor has already been included in the tree. Ifthe neighbor is not already in the tree, the system and method maycompute the XOR value (which may have already been computed fordetermining the K closest nodes). The XOR value may be considered thecost of including the neighbor node as a variation of the alreadyincluded node. If the cost is less than a predetermined threshold and isthe cheapest as compared to costs of including the neighbor node as avariation of another node of the tree, then the node may be added as avariation of the considered node. Otherwise, if there is a cheaper costfor including the neighbor node as a variation of another node of thetree (which is less than the threshold), then the neighbor node may beincluded as a variation of that other node. If the cost of adding theneighbor node as a variation to any other node is above the threshold,the neighbor node may be added separately.

Accordingly, the system and method may compute for each node an MNearest Neighbor structure with, e.g., M=100, 250, or 500. Atwo-dimensional array of nodes may be constructed, where index ‘i’ is ofthe array of all of the nodes, and index T is of the array of Mneighbors for each node[i].

The system and method may initially set each node i's cost of additionto the tree, Δ[i], as infinite. The system and method may construct apriority queue Q of nodes i according to its value Δ[i]. The system andmethod may maintain a set of trees T.

The system and method may perform the following:

For each node i:  Δ[1] = ∞; End For Δ[1] = 0; //A first considerednode's cost is initially set to 0, while all others have been set asinfinite// While there are Nodes in Q:  i = DeQueue(Q); //this meansthat i is the index of the cheapest node in the queue and the next nodeto be inserted into AMST T[t]//  if (Δ[i] > treeThresh) t++; //ifdistance exceeds treeThresh, then start new tree//  insert(i, T[t]);//insert the i^(th) node into the present tree node section//  For (j=1to M): //find M nearest neighbors of node i //   Let n_(j) be the i^(th)node's j^(th) nearest neighbor.   if (n_(j) is not in T) Do:    Let d =XOR(i,n_(j)) //Compute XOR distance between Nodes i and n_(j)//    If (d< Δ[n_(j)]) Δ[n_(j)] = d; //Update the cost of adding n_(j) to T[t] //  End Do  End For End While

Font Matching

Tangent-Based Matching

In an example embodiment of the present invention, the system and methodmay compare two nodes (also referred to as bitmaps or symbols) using atangent matcher. A tangent matcher imposes certain constraints notimposed by conventional matchers. A tangent matcher is one havingconstraints regarding a pixel's tangent, i.e., an on pixel's borderingof an off pixel. That is, a tangent matcher requires the system andmethod to analyze a 3×3 surrounding a pixel. (A feature matcher,discussed below, may require analysis of an even larger window.) In oneexample embodiment, a range of tangent matchers may be used.

When two glyphs are given to a matcher, an alignment is used so thateach pixel in one glyph has an equivalent pixel in the other glyph withthe same positioning. Generally, the glyphs will be aligned by centroid,so that the centroid pixel of one glyph will have the equivalentposition as the centroid pixel of the other glyph. Other alignments mayinclude upper-left, whereby the top-left pixels of the two glyphs willhave the same positioning. Pixels outside the bounding box of a glyphare considered to be off (or white). A matcher will require the twoglyphs to match with the given alignment. If they don't match with agiven alignment, additional alternative alignments may on occasion beused, until a match is found. If a pixel in one symbol cannot find whatit needs in the equivalent pixel of the next symbol, it may look atthose pixels which neighbor it. For example in a Hausdorff match, eachblack pixel must find a corresponding black pixel within distance one ofits equivalent pixel.

Each pixel has four 4-neighbors, namely top, bottom, right, and left. Ithas eight 8-nieghbors, which in addition to the 4-neighbors includes thetop-left, top-right, bottom-left and bottom-right. If a black pixel hasa white 4-neighbor, i.e., top, bottom, right, and/or left, it isreferred to herein as a tangent pixel. Any two or more black pixelswhich share the same white 4-neighbors may be referred to as having thesame tangent constraints. (When a black pixel contains more than onewhite 4-neighbor, we may require that all of these tangent constraintsbe satisfied, or we find it acceptable if one of these tangentsconstraints is satisfied.) A tangent matcher is one which checks whetherthe tangent pixels in one glyph can find corresponding pixels in thesecond glyph which satisfy their tangent constraints. Thesecorresponding pixels must generally be within distance one of the(positionally) equivalent pixel, and there may be additionalrestrictions as to which direction they must be in. When a tangent pixelfinds a corresponding pixel with the same tangent constraints, thelatter may be said to be giving tangent support to the former. A tangentmatcher therefore focuses on whether tangent pixels can findcorresponding pixels which offer tangent support.

For example, a weak tangent matcher may be used which determines thatfor a pixel of a first bitmap, a second bitmap includes a correspondingpixel that is a match if the corresponding pixel shares one of the sametangent directions (e.g., where a pixel has up to four tangentdirections, i.e., top, bottom, right, and left) as the pixel of thefirst bitmap. Pixels may be considered to correspond if their positionsrelative to their respective bitmaps are not offset by a certain number,e.g., 1. So, if a black pixel has a white neighbor above it and a whiteneighbor to its left, a weak tangent matcher would require that acorresponding black pixel be found in the second bitmap which shares oneof the same tangent constraints. Namely, it must have a white pixelabove it or to its left.

A stronger tangent matcher may determine a match where all tangentdirections are matched. (However, if a pixel has four tangentdirections, the pixel may be considered an isolated pixel so that nomatch is required). A tight tangent matcher may require that, if a pixelhas two tangent directions, then the other bitmap must have acorresponding single pixel with the same two tangent directions. A loosetangent matcher may find a match where two pixels of the second bitmap,which are deemed to correspond to the considered pixel of the firstbitmap include a first pixel having one of the tangent directions and asecond pixel having the other tangent direction, even though a singlepixel of the first bitmap has both tangent directions. (The tight andloose matchers may be considered sub-categories of the strong tangentmatcher.)

Example criteria for determining which tangent matcher to use includes,but is not limited to, mean stroke thickness or height. For example, thethicker the stroke the weaker the tangent matcher can be. Strokethickness can vary at different points on a considered character, sothat the system and method may apply strong and weak tangent matchers atdifferent points of the same character.

A neighbor supported matcher requires a match to be found for a pixelonly if the pixel's tangent has support by its neighbor. If the pixeldoes not have support for its tangent, then the bitmaps may bedetermined to be a match even if no match is found for the pixel.

Neighboring support is where the pixel's neighbor has a tangent in thesame direction as the considered pixel.

Nevertheless, the neighbor supported tangent may be stronger in certaininstances than the weak matcher. For example, where a pixel has atangent in two directions, but neighbor support only in one direction,then the weak tangent matcher allows a match of tangent in eitherdirection, but the neighbor supported matcher requires a match in thedirection having the support.

In an example embodiment of the present invention, the system and methodmay perform accurate shape matching that determines when an objectboundary has a well-defined tangent direction. If a tangent direction isdetected in bitmap A, then the matching function may incorporate thisconstraint, e.g., bitmap B must have a corresponding boundary point witha corresponding tangent direction.

With respect to printed text, the system and method of the presentinvention may detect the contrast between the black foreground and thewhite background of the text. Not only are the black pixels detected,but how they relate to the surrounding white pixels are also detected.For example, for a bitmap of a first letter “p” with a mean strokethickness of 2 and a second letter “p” at twice the height, width, andmean stroke of the first letter “p,” while the second bitmap is muchlarger than the first bitmap, the two underlying letters arenevertheless the same because the two bitmaps share the same tangentdirections.

Conventional JBIG2 matching relies on point set techniques which matchpixels in one bitmap to corresponding pixels in another bitmap. Whilethese techniques make sense intuitively and can be run very quickly,they don't capture the full subtlety of the human visual system. Forexample, the bitmaps shown in FIGS. 3, 5, and 7 differ from the bitmapsshown in FIGS. 2, 4, and 6, respectively, by only the few additionalpixels marked by an X. For a point set matcher, these differences wouldbe fairly insignificant and would generally not preclude a match. From atangent perspective these bitmaps would be very different, and wouldgenerally not be allowed to match.

Tangent constraints may be imposed in addition to other point settechniques, such as Quadrant Hausdorff, or they may be used instead ofthem. This may depend on the nature of the bitmaps to be matched.

Tangent based matchers create a whole new breed of matchers. Just likepoint set matching led to a range of useful matchers, so too Tangentconstraints enable a range of important new matchers. Different ones ofthe tangent matchers may be ideal for different types of bitmaps, asexplained above. Some criteria have been set forth above, and additionalexemplary criteria are discussed below.

In an example embodiment of the present invention, the system and methodmay utilize a Tangent Hausdorff matcher. A Weak Tangent Hausdorffmatcher may require that every black pixel with a white four neighbor(i.e., directly above, below, left or right) in each bitmap must find acorresponding black pixel within a distance of one that shares one ofthe same white neighbors. A Weak Tangent Hausdorff matcher is tighterthan a Hausdorff matcher, as it imposes additional tangent constraints.

A matcher which requires that every black pixel find a correspondingpixel that shares all of the same 8-neighbors may be called a T₈matcher. One that requires that every black pixel find a correspondingpixel that shares the same 4-neighbors may be called a T₄ matcher. Onethat requires that every black pixel find a corresponding pixel thatshares the same white 4-neighbors may be called a T_(W4) matcher. Onethat requires that every black pixel find a corresponding pixel thatshares one of the same white 4-neighbors may be called a T_(W1) matcher.Consequently, a Weak Tangent Matcher may be referred to as a T_(W1)matcher.

For very large bitmaps, the system and method may utilize a Rank WeakTangent Hausdorff matcher. Just like a Rank 95 Hausdorff requires that95% of the black pixels find a corresponding black pixel within adistance of one, a Rank 95 Weak Tangent Hausdorff requires that 95% ofthe black tangent pixels find a corresponding black pixel that shares asame tangent constraint within a distance of one.

For very small bitmaps, the system and method may utilize a QuadrantTangent Hausdorff matcher. Just like a Quadrant Hausdorff matcherrequires that every black pixel find a corresponding black pixel withinthe same quadrant, a Quadrant Tangent Hausdorff matcher requires thatevery tangent pixel find a corresponding black pixel with the sametangent constraints in the same quadrant.

In an example embodiment of the present invention, the system and methodmay utilize several tangent-based matching functions. They may differprimarily on how much of tangent support or a tangent match theyrequire. Further, the conditions under which the tangent-based matchingfunctions are used may differ. That is, while every tangent has a blackpixel with a neighboring white four neighbor, not every tangentdirection must necessarily be matched.

For example, a large bitmap with a mean stroke thickness greater than 4may have a single white pixel in the middle of a black region due to apoor scan. It wouldn't make sense to require the four black neighborswho neighbor this white pixel to each find tangent support, therebyrequiring a matching bitmap to have a single white noisy pixel withindistance one. Therefore, for larger shapes, the system and method mayprovide that a match is required only for that which is determined to bea structural perimeter point, which is when if the white neighbor itselfhas a white neighbor (and for even larger bitmaps that it has two whitefour neighbors.)

The tangent matcher may also differ on the degree of match required forthere to be a determination that corresponding pixels of two bitmapsmatch. For example, if a black pixel has two white neighbors, a WeakTangent Matcher would consider one or more corresponding pixels ofanother bitmap to be a match as long as the matcher finds that the oneor more corresponding pixels match the considered pixel in either one ofits two tangent directions. A Strong Tangent Matcher may require a matchfor both of its tangent directions. A Tight Tangent Matcher may requirea single corresponding pixel to match the considered pixel in bothtangent directions. The same distinction may apply to a pixel with threewhite neighbors in three tangent directions.

The tangent matcher may also differ in where the tangent match must befound. For example, some may allow any corresponding match withindistance one. Others may require the tangent match to be found in apixel that is within a certain distance of the considered pixel in aspecific direction. This will frequently be parallel to the tangentdirection. For example, a black pixel with a white neighbor above it mayrequire that its corresponding black pixel with a white neighbor aboveit of the second bitmap be either in the same exact position, or onepixel above or below it. If the only such pixel is to the right or theleft, it will not be valid support.

One type of tangent matcher is a neighbor supported tangent matcher.

Neighbor Supported Tangent Matcher

In an example embodiment of the present invention, the system and methodmay use a neighbor supported tangent matcher which requires that, if ablack pixel with a white four neighbor has an adjacent black fourneighbor which shares a white neighbor in the same direction, then thecorresponding matching bitmap must also have a black pixel with a whiteneighbor in the same direction, within distance one. For example, thebitmap shown in FIG. 8 includes a black pixel labeled X. The pixel X hasa white neighbor above it (white delineated by 0, black delineated byB), as does the black pixel to its left. The black pixel to its leftsupports the top tangent direction. Therefore, the neighbor supportedtangent matcher requires that the second bitmap also have a black pixelwithin distance one that has a white neighbor above it. There is also atangent to the right of pixel X, but, since there is no black 4 neighborsupport for that tangent, the neighbor supported tangent matcher doesnot require a tangent match in that direction.

For most variations of the Tangent Matcher, a relatively small windowsurrounding the pixel will provide all the information needed todetermine what type of support will be needed. For a Weak TangentMatcher, only the values of the black pixel's four neighbors are needed.For a Neighbor Supported Tangent Matcher, only the value of the blackpixel's eight neighbors are needed. Each of these lend themselves to aquick lookup table of acceptable pairs of windows.

Feature-Based Matching:

In an example embodiment of the present invention, the system and methodmay use a feature-based matcher. The feature-based matcher analyzes awindow larger than the 3×3 surrounding the pixel. A feature-basedmatcher is one that chooses a particular matcher based on a “feature”found by the processor in the surrounding window.

In an example embodiment of the present invention, the system and methodmay use a non-feature-based matcher, e.g., one of the tangent matchersdiscussed above, for most of a considered character, and use afeature-based matcher for certain parts of the character belonging to adetermined feature. The feature-based matcher may be used for thosecertain parts to ensure that the feature is included in the secondbitmap for a determination to be made that the second bitmap matches thefirst bitmap (even though a weak matcher may have otherwise determinedthe pixels to be a match).

To make sure the feature is included, the feature-based matcher maycompare the boundaries of the feature. For example, the structure shownin FIG. 9 may be considered to be less sharp than that of FIG. 10, suchthat the feature-based matcher may require a stricter matching criteriafor the structure of FIG. 10 than that of FIG. 9.

Features that the system may match in the two bitmaps can include butare not limited to concavity, convexity, corner, curve, cusp, maxima,minima, polar maxima, polar minima, umbilic, and inflection.

The system and method of the present invention may require that, forevery feature detected in shape A, a corresponding feature must be foundin shape B.

One way to view a feature-based matcher is as a sophisticated hybridtangent matcher. While they may each have different rules, a tangentmatcher may require the same level of tangent match for every tangentpixel in the bitmap. A feature-based matcher may classify each blackpixel with a white four neighbor into different feature classes, each ofwhich may require different degrees of tangent match in differentdirections.

For example, while the black pixels marked X in FIGS. 9 and 10 eachshare the same tangents, the human visual system treats them verydifferently. The feature-based matcher of the present invention may takethis into account. A feature-based matcher would allow the X in FIG. 9to find a match to a corresponding pixel of a second bitmap having thesame tangent in either direction, while the feature-based matcher mayrequire the X in FIG. 10 to find a match only in a single pixel havingboth tangents.

Curvature-Based Matching

A curvature is wherever there is a consecutive directional shift in asame direction within a threshold number of pixels. In an exampleembodiment of the present invention, where there is such a shift, thesystem and method may require the same number of directional shifts inthe same direction in the second bitmap within a tolerance of 1 for thesecond bitmap to be considered a match to the first bitmap. Thetolerance of 1 may apply with respect to (a) the number of shifts and to(b) the position of where the shifts occur.

For example, as regards (a), if the source has 2 shifts, the destinationcan have one shift. As regards (b), the shift in the second bitmap canbe one above or below the corresponding shift in the first bitmap if itthe shift is a horizontal one (and right or left if it is a verticalone).

The feature-based matcher may require that if the discrete curvature ofshape A is locally well-defined then so must shape B have correspondinglocally well-defined curvature that matches the curvature of A. Asequence of tangent directions on the boundary of A must have a matchingsequence of tangent directions on the boundary of B. While this is asubcategory of a feature matcher, this can be viewed as a higher leveltangent or feature matcher. It may not be sufficient for each localfeature in A to find a corresponding feature in B. Rather, a set ofneighboring features must all find supporting features that have thesame relative positioning to each other. Alternatively, a set ofneighboring tangents must all find tangent features that have the samerelative positioning to each other.

For example, the 3 neighboring X pixels in each of FIGS. 14 and 15 havetangent support to the right, and if they were aligned properly each ofthe X pixels of one of the bitmaps of FIGS. 14 and 15 would find a matchin the other of the bitmaps of FIGS. 14 and 15 within distance one.However, the relative positioning between the 3 Xs are very different ineach of the bitmaps, and a curvature-based matcher may therefore rejectthe match.

Feature-based curvature matching is generally a way of implementing afarther reaching feature-based matcher. Other than requiring thefeature-based matcher to analyze an overly large window, thefeature-based curvature matcher may analyze a series of smaller windowsand insist on maintaining the relationship between them.

Feature-Based Symmetry Matching

One kind of feature the feature-based matcher may consider is symmetry.In an example embodiment of the present invention, if a concavity orconvexity is found, the system and method may determine whether theconcavity or convexity is noise or, otherwise, significant. If thesystem and method determines that there is reflexive symmetry along theaxis, the feature may then be considered significant.

The system and method may always look for symmetry (unless there is anextremely large stroke thickness). Whenever there is symmetry, thesystem and method may require a tangent match in all directions, (butwill not require it by a single pixel, so that the second bitmap mighthave an X below the corresponding X besides the corresponding X and maystill be considered a match.)

With respect to concavity, if a symmetrical concavity is found, thesystem and method may require the same in the second bitmap, and tangentmatching would not necessarily be that which is used to ensure such amatch. It may occur that the tangent matcher would find a match wherethe feature-based symmetry matcher does not.

A feature matcher will frequently require the matcher to look at a muchlarger window than the immediate eight neighbors of the bitmaps. Aneffective feature matcher has rules to know when a larger window isneeded, as well as what shape and in which direction it must be in. Thesystem and method may provide that the feature matcher search forcertain precise predetermined properties in that window. While the humanvisual system regularly takes such information into account, the binarycompression community has not taken such information into account.Instead, conventional systems stick to small local windows. Furthermore,considering such information is too tedious to be effectively solvedusing conventional systems. However, an example embodiment of thepresent invention provides for considering symmetry to take into accountinformation of a large window.

If a structure (weaker than a feature) is detected in symmetry in shapeA, then, even if based on local criteria there wouldn't appear to be aneed for a feature match, the matcher may require a correspondingsymmetric structure in shape B.

For example, FIG. 11 shows a thin vertical line with a stroke thicknessof two, which has an extra black pixel on the right side. While all thepixels on the right side have tangent support to their right, this pixelalso has tangent support above and below it. Considering the X pixel,without addressing it as a feature, a matcher would not require acorresponding pixel to also have tangent support above and below. On theother hand, FIG. 12 shows the same line except that the single X pixelhas its tangent directions of above and below reinforced by asymmetrical pixel on the left side of the bitmap. Consequently,feature-based symmetry matching would require that both symmetricalpixels find a matching pixel tangent in a second bitmap for the aboveand below directions.

This rule requires the use of a much larger window than most matcherswould consider. For example, with respect to the pixel marked X in FIG.12, the white pixels which are four pixels to its left must beconsidered, as shown in FIG. 13.

Word Grouping, Learning, & Matching

Letters may get stuck together. Using a precise matcher, it maytherefore occur that no match will be found for these letters. In anexample embodiment of the present invention, the system and method mayconsider whole words as a unit and match words even where the componentletters would not otherwise be separately matched.

The system and method may determine that a string of components is aword. To do so, the system and method may determine where there arelines of characters. For example, the system and method may search forblack pixels with white pixels underneath the black pixels and count thenumber of such pixels in each row. The system and method may determinewhere the bottom of the line is based on peaks of the counted pixels atcertain rows. The system and method may similarly count those blackpixels having white pixels above for determining the tops of the lines.(Where the line is a vertical one, right and left, rather than top andbottom, may be considered.)

The processor may measure the distance between every two components onthe line. The processor may then determine which two adjacentcomponents' distance from each other is closer to an intra-word distanceand which is closer to an inter-word distance. Those closer tointra-word may be determined to be of a same word. Others may bedetermined to be of separate words.

For each word, the processor may set a bounding box. Words may beconsidered candidate matches if they have a match of the bounding boxes,e.g., within two for height, width, and area of black pixels that areon. Depending on the stroke thickness, one of the matchers are run. (Ifthick enough then it is not a tangent matcher; if thinner, a the tangentmatcher may be used, as discussed above.) However, when matching words,the processor may allow a match with some mismatch in certain positionsdetermined to be a candidate for where two letters are connected.

To determine which positions are candidates, for every column (assumingthe lines are horizontal), the system and method may count the number ofblack pixels that are on. If there is a minima (very few black pixels),e.g., as compared to other columns, then it is a candidate for havingtwo letters stuck together at the column of the minima or within onecolumn of the minima.

Word Groups

Conventional matching is done without merging or breaking nodes. Thislimits compression. Also, for other applications, such as OCRpropagation, not learning across topological boundaries is a limitingfactor. Word topology is highly reliable, meaning that segmenting wordsinto word groups can be done with a high degree of reliability,regardless of the capture resolution, even at handheld device (e.g.,cell phone) image capture rates. If word instances can be groupedreliably, then the system and method can learn word groups, meaningwords that re-occur with high frequency, even if the topology of wordinstances within the word group is not constant, e.g., where the lettersin one word bitmap are stuck together and the letters of another wordbitmap are not stuck together.

Learning Word Classes

Conventional bitonal compression shape matchers do not match shapes withdiffering topology. Rather, they match single components against similarsingle components. In an example embodiment of the present invention,the system and method may implement a word matcher that does matchacross varying topology, thereby facilitating achieving human perceptualperformance for learning the underlying font library and for propagatingOCR-based information and constraints.

Piecewise TP Matching:

In the following, a component for which there is no match is referred toas a “residual component.” The system and method of the presentinvention may allow such residual components and still consider thewords to be a match if the residual components occur where there is aminima of black pixels.

Alternatively, for all of the residual pixels, the system and method maymark as single components each set of connected residual pixels. If theformed components are too large, i.e., the component includes more thana threshold number of pixels, then the system and method determines thatthere is no match. Otherwise, the system and method may perform thefollowing:

determine whether the residual component has a width greater than apredetermined number of columns, e.g., 3, a match not being found if itis greater than the predetermined number of columns;

if the width is not greater than the predetermined number, count thenumber of consecutive columns each having within the column (in theconsidered line) one or more residual pixels and determine whether thenumber is greater than a predetermined number (which may be the same ordifferent as the predetermined number of the previous step), in whichcase the words are not deemed to be a match;

if the number of consecutive columns is not greater than the secondpredetermined number, then determine whether separate occurrences ofresiduals occurs within a predetermined number (e.g., a thirdpredetermined number) of columns, e.g., 8 columns;

if they occur within the predetermined number of columns, then thesystem and method may deem the words not to be matches, unless theseparated residuals completely occur within a very small number ofcolumns, e.g., 3 columns;

if they do not occur within the predetermined number of columns orotherwise occur completely within the small number of columns, then theresidual components may be disregarded, and a match may be found despitethe residual components.

Accordingly, the system and method may match two words as follows: treateach (multi-component) word as a “component” and use a Hausdorffdistance 1 with tangent constraints as the metric; mark all pixels thatdo not match using this metric as residual and perform residualconnected components on these pixels; if any residual connectedcomponent has an area exceeding a max then reject word match; otherwise,define a band image as an image with 3 columns and height equal to theword height; find a minimal set of bands that covers the residualconnected components (min band cover); if these covering bands have aminimal distance dmin from each other, e.g., dmin>=8, then the words areallowed to match, else they are not; if the words match, theircorresponding models are allowed to match.

The compression system and method may learn word groups and all thecorresponding models within a word group may be merged together. Thisalso allows the system and method to break component models where someword instances have separate models and some instances havetopologically connected models.

Word Grouping: 2 Cases

i. Where the Text Direction is Known.

The system and method of the present invention may group page so thattext direction is running horizontally and sweep page from left to rightinto text segments. These basic line groupings can be extended byconnecting line segments together. Once a line segment has been found,the interletter and interline distances may be solved for. With thesetwo distances d1 and d2 computed, the line segment can be divided intowords.

ii. Where Text Direction is Unknown.

The system and method of the present invention may perform word groupingin a manner that does not depend on line finding. First, for eachcomponent, its Cartesian k nearest neighbors may be found, where ‘k’ istypically 5 and nearest neighbor is measured using Euclidean distancebetween corresponding centroids. A grid may be overlayed on the image,where each grid square is a pointer to a linked list. In this way, eachcomponent is put onto a linked list. A component's nearest neighbor (NN)components may be found by checking the considered component's list andneighboring grid lists.

Once a component's NN is found, the components may be linked together iftheir positions are sufficiently close to each other. This usually meanstheir respective bounding boxes are within a few pixels. When NNs areallowed to link they form a subline. Additional connected components canlink to a subline if this link is consistent in direction (orientation)and interletter distance with the existing subline.

Font Matching Speedup Using AKNN Data Structure

AKNN is an array of nearest neighbors for each node as described above(built, for example, using XOR). Although the XOR and moment-basedmethods of matching may have been performed, these matching methodsgenerally do not perform as an exact a matching as do the matchers.Therefore, even after finding nearest neighbors, the system and methodof the present invention may run a matcher on the nearest neighbors.

The first nearest neighbor may have already found a matching model ormay be a model. Therefore, the system and method may initially attemptto match the considered node to the model with which its nearestneighbor is associated. If the neighbor was not already matched to amodel and is not a model, then the system and method may move on to thenext neighbor. This is a routine that should have the speed of the firstfit but accuracy of a best fit methods for finding a matching model.That is, this method may increase efficiency, since the likelihood offinding a match with one of the initially considered models isincreased.

Accordingly, an AKNN data structure may be generated before attemptingfont learning.

Instead of conventional first fit matching or best fit matching thesystem and method may rely on nearest neighbor matching. Thus, thesystem and method may first compare the given node or model to itsnearest node or model to see if it matches. If it does not, the systemand method may compare to the 2^(nd) nearest neighbor.

The AKNN data structure (ds) is originally computed for the nodes (orcc's). It can then be iteratively recomputed after each matchingiteration for the current set of models.

Font Consistency Checking

In an example embodiment of the present invention, where a node'snearest neighbor is of a different model than is the presentlyconsidered node, the system and method may form a new model based on thepresent node, as follows.

For Font Consistency Checking, the system and method may run regularMatcher( ) where all nodes are matched to a model. Using AKNN, thesystem and method may determine font consistency for each model M. Amodel M is defined as consistent exactly when for each node associatedwith the model, its k nearest neighbors (NNs) are all associated withmodel M, for some k, e.g., k=3, k=5, or k=10. (Alternatively, k may becomputed as a percentage, e.g., 30%, of the number of nodes associatedwith the model M.)

Where j is index through number of models, for every single model, i isan index through all nodes in a given model (a class of nodes that maybe slightly different but have been matched and are grouped together toform a model), and N is the number of models, then if node n[i] in modelM[j] has a NN in model M[k], and j!=k, then:

N++; // add to the number of models.// let n[i] form basis of new modelclass N; for all nodes l in M[j]  if (cDist(n[l], M[N]) < cDist(n[l],M[j])   ClusterMove(n[l], j,N); //if the node of the previouslyconsidered model is actually closer to the newly for all nodes l in M[k]model, then change the model to which it belongs.//  if (cDist(n[l],M[N]) < cDist(n[l], M[k])   ClusterMove(n[l], k,N);

This font consistency check may avoid mismatches.

Font Model Reduction Using AKNN

In an alternative example embodiment of the present invention, if anode's nearest neighbor is of a different model than the considerednode, the system and method may join the two models together, instead ofcreating a new model. Whether to use one or the other can be set by theuser, but the creation of a new model is safer but more expensive.

With respect to this second approach, there are some safety measures asto when to apply it. For example, a safety measure may be to compare thetwo models themselves to each other using a very loose matcher andrequire to find a match. Alternatively, a safety measure may be torequire that the nodes themselves match using a regular matcher formerging the models.

Accordingly, the system and method may use computed fast approximatenearest neighbors (AKNNs) to reduce font models. Specifically, if fontmodel p includes a node j whose k^(th) nearest neighbor 1 belongs tofont model m then merge models p & q. Alternatively, merge models p andq assuming the Hausdorff distance H(p, q)<=1 or H(p, q)<=2.Alternatively, merge models p and q assuming nodes j and 1 match eachother with the regular matcher.

The method for reducing font models using approximate K nearest neighboris as follows (in the following k_nn[ ] is an array where k_nn[i,j] isthe node index of the i^(th) node's j^(th) nearest neighbor):

  For each node i in a given image: set k=10; {   for j = 1 to k {     if ( p = model_class(node[i]) and q = model_class(node[k_nn[i, j]])and p<>q )    if H(p, q) <= d // d is typically 1 or 2     merge_models(p, q);   } }

Method for Reducing Number of Font Models Using Cartesian Product

In an example embodiment of the present invention, the system and methodmay first match nodes to create models and then match each created modelto all other models. If they are matched, they are merged; otherwisethey remain separate models. Then the system and method may match eachnode of each of models determined to be relatively close to each other(but not determined to have been matched) to the nodes of the other ofthe models determined to be relatively close to each other. If they arematched, then the models are joined together (even though the modelsthemselves did not match each other).

The Cartesian product approach is meant to reduce the fragmentation in afont model set under construction. The Cartesian product of two sets isused here to identify possible model merging that should occur. Inparticular, for example, where the system and method is in the processof font learning and currently have n font models, then a method tofurther reduce the number of models is as follows:

For i = 1 to n //iterate through all models  For j = 1 to n //iteratethrough all models  If (i != j)   if H(i, j) <= d // d is typically 1 or2    For k = 1 to num_nodes(i)     For l = 1 to num_nodes(j) { //iteratethrough cartesian product of nodes     Node_index1 = Model_Node(i, k)//returns index of k^(th) node belonging to i^(th) model     Node_index2= Model_Node(j, l) //returns index of l^(th) node belonging to j^(th)model      If Node_Match(Node_index1, Node_index2)       merge_models(i,j);

Maximal Entropy Cluster Probing (MECP):

If a conventional system is working on page 2 of a document and hasalready solved everything in the first page, font learning is performedagain in the second page anew. The following is a new fast way ofprocessing a page 2.

In an example embodiment of the present invention, the system and methodmay compare each character of the second page to existing models anddetermine to which model, if any, the character belongs.

The system and method may use hashing for this purpose. Hashing is, forexample, where a value is entered into a function and a remainder numberof the result of the function is recorded—only a few records would bemapped to a particular hash value. Therefore, a new entry need bechecked only against a subset of all of the existing records. In thecase of the example embodiment of the present invention, the system andmethod may use a hashing function to match the characters of the secondpage to the models generated based on the first page.

The following may be used as the hashing function:

All models that have been generated may be stacked. Figurativelyspeaking, a pin may be entered into the models to determine which pixelsvary the most between the models. If a pixel is almost always whitethrough all models, then the system and method may ignore it. The closerit is to 50/50, the better it is for the system and method to consider.Clusters may then be formed on the basis of such pixels. For example, if50% of the stacked models are black with respect to a particular pixel,then that pixel may be used to narrow the number of possible modelmatches to 50% of the total. Then with another pixel, the system andmethod may break the selected cluster down to further clusters. Thesystem and method may keep probing until one model remains.

(Note that some pixels of the models are gray which means that the pixelvaries between the nodes of the model. Gray pixels of the model maytherefore be ignored.)

First the probes to be used are determined. Then for each character wethe probes are applied to do the narrowing down.

Once the one model to which it can belong is determined, if it actuallybelongs to the model may be determined by looking at the gray scalemodel having black, white, and gray. The processor may determine thatthe node matches the model if all on pixels (black) of the node map toon pixels or gray pixels of the model, and all off pixels of the nodemap to off or gray pixels of the model.

Sometimes, where there are not enough nodes to rely on the grayscalemodel, then the following may be performed: mark all black boundarypixels and all white boundary pixels as a gray pixel. A black boundarypixel is one that borders on a white and a white boundary pixel is onethat borders on a black. Then the system and method may do the same asabove.

A Maximal Entropy Model (MEM) may be used to determine which pixels formthe smallest set of pixels that can be used to probe the models toalways narrow down to one.

Accordingly, it is desirable to establish a set of pixels such that thesize of the pixel set is at a minimum. The set points are given by their(xi, yi) geometric locations, every pixel in the set assigned a valueV(xi, yi) of either ON or OFF, and for a given learned font, eachBoolean assignment uniquely identifying a digitized symbol in thealphabet.

Purpose of maximal entropy coding: To find a hashing function that, insome constant time, maps a bitmap to a page-based or global fontlibrary. Typically, hashing functions are not used in the imagingdomain. Even though hashing functions are very useful in other areas,like fast string searching, the imaging domain is non-deterministic sothat hashing is typically very difficult.

In the case of maximal entropy coding, a grayscale library is firstcomputed, consisting of all font bitmaps in the given regions (e.g.,this page, across k pages). Initially, all the font symbols (i.e.,bitmaps) are in one cluster. At each iteration the system and methodgoes to the largest cluster and finds a pixel, white or black but notgray, such that it breaks the cluster into as even a set of subclustersas possible. For an alphabet of k symbols, no more than k−1 probes arerequired. This set of probes finds a minimal set of pixels withrespective Boolean assignments of ON and OFF that uniquely identifiesevery symbol in the font library. This minimal Boolean assignment is thefont hashing function that can be used both for fast matching and fastASCII OCR assignment.

Fast Font-Based Image Hashing

Hashing is a technique that has been used extensively in dictionary anddatabase construction, but has not been used extensively in computervision and image processing due to the non-deterministic nature ofimaging and document capture.

In an example embodiment of the present invention, image hashing may beused as an effective tool for compression, OCR, and other applications.A resolution is selected across the document, e.g., 300 dpi. For thisresolution, it is assumed that most picture objects will preserve theirshape under basic morphological operations of erosion and dilation. Thepurpose of this method is to find similar image objects and can beapplied to all kinds of image objects including fonts, logos, etc.

After processing k pages of a document, close to the complete library ofpicture objects, or actually the complete library of picture objects,for this document may have been computed. Nevertheless, there may besome additional picture objects, but it is not known when or where theymight occur.

The hashing function may be perfect, that is collision free, and alsominimal if possible, where the size of the hash table and font libraryare both m. A minimal perfect image hashing function can be constructedas follows:

Assume a perfectly calibrated imaging environment. Then each node ormodel instantiation must be within a Hausdorff 1 distance of any othermodel instantiation. Let us assume that a well-digitized, static libraryresolution can always be found that is topologically well-behaved (TWB).A library is TWB whenever the erosion of foreground does not change thetopology of the foreground and, similarly, for the background. If alibrary violates this assumption, the resolution of all picture objectsin the library may be doubled until this assumption is satisfied.Alternatively, a library is geometrically well-behaved GWB whenever theerosion of foreground is a Hausdorff distance 1 match with the originalforeground and, similarly, for the background.

So having normalized the original font library so that it is TWB(alternatively, GWB), a minimal set of image points that discriminatebetween each element in the library may be found. Such a set of pointscan always be found provided the interior (i.e., a black pixelsurrounded by black pixels) and exterior (i.e., a white pixel surroundedby white pixels) points for each font image are distinct. Thisdiscriminating set of points is the image hashing function for thelibrary. Since the size of the alphabet is m, and the number of binslots in the hashing function is m, and the function is always collisionfree, the hashing function is minimal and perfect.

Once the font library has been hashed, we can process the document,identifying each node's possibly matching model in constant time. Thisis also useful for very fast OCR. For each node, use the hashingfunction to identify the model class. Then reference the model in JBIG2coding, or propagate the OCR mapping. It is assumed that when thelibrary is static, no new models need to be learned.

Fast Model Verification Method For Hashing-Based Global Model Matching

Matching on a page once a font library is known is very different frommatching when a font is being learned dynamically. Let's assume that formost documents the font can be learned in a fraction of the total pagesin the document.

Using maximal entropy cluster probing (MECP), we can hypothesize that aconnected component on the current page belongs to a certain font model.For each global font model, we have computed a tri-value grayscale map,where each pixel is ON, OFF, or gray. If a pixel is ON, then all nodeinstances of this model have this pixel turned ON. If a pixel is OFF,then all node instances of this model have this pixel turned OFF. If apixel is GRAY then some node instances matching this model have thispixel turned OFF, while other node instances have this pixel turned OFF.

To test (verify) that this node matches to a given global font model wecheck each node pixel against the grayscale model image. Each pixel ONin the node image can map to either an ON or GRAY pixel in the grayscaleimage, but not to an OFF pixel. Similarly, each pixel OFF in the nodeimage can map to either an OFF or GRAY pixel in the grayscale image, butnot to an ON pixel.

Fast Verification Of Page Node Against Global Or Page Grayscale FontModel

it is desirable to use as few computer cycles as possible. In additionto the grayscale, the system and method of the present invention maycreate an on image map and off image map. In the on map all the gray isassigned an on value; in the off map, all the gray is assigned an offvalue. So every model will have an additional two models—on and off Thesystem and method may use these two maps for the comparison of the nodesto the models as set forth below.

Using MECP, we can hypothesize that a node is a digitized instance of afont model. But testing of this conjecture is necessary. To that end,each font model has a grayscale image map that has been computed. Inaddition to the grayscale map, we also compute an ON map and an OFF map.The ON image map for a given model is binary, and is stored as an arrayof 1D row arrays or vectors. To compute a model's binary ON image, eachON or GRAY pixel in the grayscale image is assigned a binary ON value inthe binary ON map. To compute a model's binary OFF image, each OFF orGRAY pixel in the grayscale image is assigned a binary OFF value in thebinary OFF map.

Verification of a row of pixels in a node image consists of two steps:

-   -   i. all ON pixels in the node vector must be covered with        supporting ON pixels in the binary (thresholded) ON image, and    -   ii. all OFF pixels in the node vector must be covered with        supporting OFF pixels in the binary (thresholded) OFF image

To verify the ON pixels in a row of the node image, the row is ANDedwith the binary ON row computed from the grayscale model image. Thecurrent node vector (row) ANDed with the binary ON row equals thecurrent node vector exactly when the binary ON row covers all the ONpixels in the node row, which is step i. above. This verification can bedone, for bitmaps with up to 64 columns, in a single clock cycle.Similarly, to verify that the OFF pixels in a row of the node image arecorrect, the node vector is ORed with the corresponding row of thebinary OFF map. The current node vector ORed with the binary OFF rowequals the current node vector exactly when the binary OFF row coversall the OFF pixels in the node row, which is step ii. above. This secondverification can be done, for bitmaps with up to 64 columns, in a singleclock cycle. Total verification time is on the order of two clockcycles.

If the node does not match using this test, then we run a regularmatcher for the node to the model. If it passes, then the model isupdated to account for the new node, and the node is set to belong tothe model.

If it fails the matcher's test, then the node is compared to other nodeson the page and assigned to the model of the node to which theconsidered node is matched.

If there is no way to join a node into an existing model, then a newmodel will be made.

Method For Computing Global Font Models

For multipage documents, additional compression is attained using globalmodels. This allows sharing font models across pages. The method forobtaining global models is to have a dynamic global font library. Oncethe font models for a given page have been computed, these models arematched against the current global font library. Each page model thatmatches a global model is removed from the page models and thecorresponding global model is referenced instead. If a specific pagemodel is not in the global model library (i.e., does not match a globalmode) but has sufficient frequency on this page or other characteristicsof a well-behaved font model, then it is added to the global fontlibrary. Characteristics of a well-behaved font model can include, butare not limited to, the height, width, area or median stroke size ofthis model being closely aligned with corresponding values of otherknown font models.

Method For Computing Composite Font Models

Use of word modeling for matching purposes has been described. Further,in an example embodiment of the present invention, some composites ofnodes may be used as an additional model for the compressedrepresentation of the input file. For example, “ing” or “th” might besuch composites. The following is an algorithm for determining whichcomposites are worth generating as a new model in view of its savingoverall characters being sent over.

Symbol addressing within JBIG2 is not that efficient. Thus, many bytescan be taken up in addressing within the symbol stream. One way toreduce addressing costs is to form composite models, thereby reducingthe number of symbols in the addressing stream. Composite models areefficient when models repeatedly occur together in a certain sequence.These composites can be formed using a greedy algorithm, as describedbelow.

Find the most common co-occurrence of models in the symbol stream withthe same relative pixel positioning. (In one embodiment, a smalltolerance of one pixel is allowed in the vertical or horizontaldirections). If that number of co-occurrences is above a certainthreshold, create a new composite model that consists of a reference toeach of those two symbols with their relative positioning. Replace allinstances of these two models that are within a tolerance of theirrelative pixel positioning in the symbol stream. Treat this newcomposite as any other symbol, and proceed to find the new most commonco-occurrence of models in the symbol stream. Iterate until the mostcommon co-occurrence is no longer greater than the replacementthreshold.

Method for Computing Global Composite Font Models

There are times when the composite model should be made available forreferencing locally on the present page and then other times where itmakes sense to include it in a global table of models for referencing byother pages. (The cost of entering in the global table after thecomposite has in any case been sent is that the index size of the globaltable is increased). The benefit of referencing the same globalcomposite by other pages rather than for each separate page may beweighed against the cost of the increased index size. Thus, the systemand method of the present invention may determine whether the benefit ofincluding the composite in the global table outweighs the cost. The waythis is determined may be by determining that the number of times thecomposite occurs across all pages is above a threshold. However, even inthis case, the system and method may refrain from including it in theglobal table, unless each component model of the composite has also beendetermined to be included in the global table. For example, for “th” tobe included, it may be required for there to have been a determinationthat “t” and “h” should be included in the global table.

Just as composite models can reduce addressing costs when appliedlocally, so global composites can also reduce per page addressing. Themethod for computing global composite models is very similar tocomputing page-based composites. The algorithm is also greedy. It relieson a 2-pass approach, where after the 1^(st) approach the global fontmodels for the document are known. In the 2^(nd) pass, global compositesare constructed in a greedy manner. The method for building up theglobal composite dictionary is the same as the one used for localcomposites, but the co-occurring symbols must both be global models orcomposites. Additionally, we may require that the global compositesappear on more than one page of the document.

Context-Sensitivity

Model matching is generally done with respect to two models A and B.Additional compression is achieved using context-sensitive matching.

Two Context Sensitive Methods:

1. If two models almost match then we consider “context.” If 2 modelsalmost match and they share intra-word left or right models, then thesystem and method may allow the models to match using a looser matcherand merge their models. Depending on the properties of models, “almostmatch” can mean within a tangent-based Hausdorff distance 1 undertranslation, a neighbor supported tangent match, or some other matcherlooser than that which would otherwise have been used on these twomodels.

Alternatively, it may be required that the intra-word left or rightmodels each have the same relative vertical and horizontal positioning.This means that if in one case the neighbor to the right is x number ofpixels away and in the second case it is fewer number of pixels awaythen, if the difference is above a certain threshold then the system andmethod would not apply the looser matcher.

Alternatively, it may be required that the models share both intra-wordleft and right models.

Alternatively, the number of models they share to the right or to theleft may help determine which looser matcher they must satisfy in orderto be merged.

2. In a 2^(nd) implementation of context-sensitive matching:

In an example embodiment of the present invention, strings of apredetermined size may be matched to all strings of the same size, e.g.,on a page, using a looser than regular matcher. For example, if asubstring str1 has 3 consecutive image nodes u1 v1 w1 in a document, anda 2^(nd) substring str2 has 3 consecutive image nodes u2 v2 w2 in thesame document, where both str1 and str2 are intra-word strings, then ifstr1 and str2 have a match, e.g., component-wise tangent-based Hausdorffdistance 1 under translation (an exemplary looser matcher), the systemand method of the present invention may then merge correspondingcomponent models P1 Q1 R1 and P2 Q2 R2.

Improvements in Bitmap Encoding Efficient Lossless Encoding UsingApproximate, Min Spanning Forest or Approximate K Nearest Neighbors

When encoding any given node in the compressed file representing anoriginal input file, the coding provides for analyzing the neighboringpixels to determine a probability of the pixel being black or white; thegreater the probability is one way or the other, the fewer bits requiredfor the encoding. Further, if there is a similar bitmap that can beused, then the probabilities are further enhanced based on the similarbitmap. But if a bitmap which is very dissimilar is used, then use ofthe bitmap can have the reverse effect. Accordingly, in an exampleembodiment of the present invention, the system and method may determinewhether a bitmap is close enough such that it can be expected on averageto save bytes. In this regard, the system and method may account forthat a saving for the presently considered bitmap might actually cause aloss for other bitmaps.

In this regard, JBIG2, for example, allows clairvoyant encoding, wherebya node may be encoded with the context of its own already encodedpixels, as well as the pixels of another already encoded node. When thetwo nodes are similar to each other, such encoding can be cheaper thanencoding the node explicitly (i.e., without the help of a second node).However, if the nodes are different enough from each other, encoding thenode clairvoyantly can increase file size. While, for each node, thesystem and method may find the closest node already encoded in an XORsense, the system and method may further provide a fast heuristic todetermine whether to encode a respective node using that nearestneighbor or otherwise explicitly.

Accordingly, the system and method may count the number of XOR pixelsbetween the considered node and its nearest neighbor that has alreadybeen encoded. Subsequent to the counting, the system and method may addto the total: (a) a multiple of the absolute difference in heightbetween the two compared nodes; and (b) a multiple of the absolutedifference in width. The system and method may determine whether the sumis less than a certain predetermined percentage of the number of pixelsin the node to be encoded (the total number of pixels calculated, e.g.,as the width times height of the node). If it is determined that the sumis less than the percentage, then the system and method may concludethat the node is to be encoded clairvoyantly, using the other node as areference. Otherwise, the system and method may conclude that the nodeis to be encoded explicitly with no such referencing. It is noted thatin lossless encoding, each node is encoded, rather than models of thenodes.

A preferred embodiment of this is to use a Min Spanning Tree (MST), andto encode the nodes in the order in which they joined this MST. Eachnode may then be encoded with its parent node in the MST serving as areference node, if it passes the test explained in the previousparagraph.

Improvements in JBIG2 Addressing Vertical Padding

When encoding glyphs in the compressed representative file, the systemand method may include in the file a set of models and indications ofwhere the models are to be placed.

For example, vertical position of the model may be encoded by anindication of the position at which the bottom-left of the bitmap is tobe placed. (Variations may be used in which other positions of thebitmap are used. The vertical padding may be similarly applied when thevariations are used. Similarly, while vertical positioning is described,the same principles may be applied to horizontal positioning, e.g.,where glyphs are arranged in vertical lines.)

Conventionally, where a line of glyphs include a plurality ofconsecutive glyphs having the same vertical position, for efficientencoding, the vertical position of the first glyph is provided with anindication that it is to be applied to following glyphs as well.Further, in an instance where a word, such as “merge,” having charactersof different height, is encoded, two alternative encoding methods areused. According to one method, using the example of the word “merge,”the vertical value of the first character would be indicated, then theshift down for the letter ‘g’ would be indicated, and then the shiftback up to the regular line height of the majority of the letters wouldbe indicated. According to the alternative method, the symbols will beordered so that whenever possible, consecutive symbols will have thesame vertical position. Therefore, those symbols having the firstvertical position would be given, followed by those with the secondposition. For example, the coding would first provide the verticalpositioning of “mer e” temporarily skipping the ‘g’ and returning to itlater.

In an example embodiment of the present invention, the system and methodmay further increase efficiency in such an instance of a string ofglyphs having different heights. The system and method may increase theheight of the bitmaps of the shorter characters, e.g., with additionaloff pixels, so that the bitmaps have the same height as the tallerbitmaps. For example, in the case of the word “merge,” the system andmethod may increase the height of the bitmaps for the letters ‘m,’ ‘e,’and ‘r’ (the bitmap of ‘e’ is referenced twice). Accordingly, the samelower-left vertical position may be used for all of the glyphs of theline. Various criteria may be used to determine when such size increaseis more efficient that an increased number of vertical positionencodings. For example, where the number of additional bytes used forthe increased bitmaps is more that than the number of additionalposition references, then it may be determined that the increasedbitmaps is not to be used. The ratio of glyphs to models may be used asa heuristic to determine whether vertical padding can be expected tosave bytes.

A descendor is a symbol whose bottom position is below that of othersymbols on line, such as ‘g’ or ‘p.’ An ascendor is a symbol whose topposition is above that of other symbols on line, such as a capitalletters or numbers. When the symbols are encoded using the bottom-leftor bottom-right, the blank rows of padding will be applied to the bottomof the bitmaps, giving all the symbols on the line the same bottompositioning of the descendors. When the symbols are encoded using thetop-left or top-right, the blank rows of padding will be applied to thetop of the bitmaps, giving all the symbols on the line the same toppositioning as the ascendors. Even when it is not possible for everyglyph on the line to have the same exact vertical positioning, the useof vertical padding can save bytes by reducing the range in verticalpositioning between symbols.

In this regard, considerable byte cost in JBIG2 and any similar bitonalcompression codec is required for addressing. This cost can be reducedby padding the letters so that linear text, incurs less of a verticalshift cost between consecutive symbols, which can often occur whendescenders are adjacent to non-descenders. An advantage of padding afont to uniform vertical height is that the vertical shift addressingcost is reduced.

In fact, it can help reduce horizontal addressing costs as well.Normally in JBIG2 coding it is required to encode the full horizontaldistance from the corner of one node to the same corresponding corner ofthe next node, so that the full distance of a node plus the interletterdistance between the two nodes must be encoded. However, if two adjacentnodes have the same vertical position, JBIG2 allows for encoding theinterletter distance between the two nodes, which in addition to beingsmaller is likely to have a more uniform value, and is therefore cheaperto encode. Enabling more nodes to have the same vertical positioning cantherefore also reduce the horizontal addressing cost of the JBIG2encoding.

An example embodiment of the present invention is directed to aprocessor, which may be implemented using any conventional processingcircuit, e.g., a Central Processing Unit (CPU) of a Personal Computer(PC), to execute code provided, e.g., on a hardware-implementedcomputer-readable medium, to perform the example methods describedabove, or portions thereof. The hardware-implemented computer-readablemedium may be implemented as a memory device, which may include anyconventional permanent and/or temporary memory circuits or combinationthereof, a non-exhaustive list of which includes Random Access Memory(RAM), Read Only Memory (ROM), Compact Disks (CD), Digital VersatileDisk (DVD), and magnetic tape.

An example embodiment of the present invention is directed to ahardware-implemented computer readable medium having stored thereoninstructions, which, when executed, cause a processor to perform theexample methods described above, or portions thereof.

An example embodiment of the present invention is directed to a methodof transmitting instructions executable by a processor, theinstructions, when executed, causing the processor to perform theexample methods described above, or portions thereof.

Those skilled in the art can appreciate from the foregoing descriptionthat the present invention may be implemented in a variety of forms, andthat the various embodiments may be implemented alone or in combination.Therefore, while the embodiments of the present invention have beendescribed in connection with particular examples thereof, the true scopeof the embodiments and/or methods of the present invention should not beso limited since other modifications will become apparent to the skilledpractitioner upon a study of the drawings, specification, and followingclaims.

1. A symbol matching method, comprising: determining which pixels of afirst symbol are tangent pixels; for each of the determined tangentpixels, determining whether a second symbol includes a pixelcorresponding to the tangent pixel that includes at least one sametangent constraint as that of the tangent pixel; accepting the first andsecond symbols as a match based on shared tangent constraintsconditional upon a determination that the second symbol includes foreach of at least a subset of the tangent pixels of the first symbol acorresponding pixel that includes the at least one same tangentconstraint as that of the tangent pixel; and generating a documentincluding a single symbol that is mapped to the first and secondsymbols.
 2. The symbol matching method of claim 1, wherein the acceptingof the match is conditional upon that, for each of the at least a subsetof the tangent pixels of the first symbol, a corresponding pixel of thesecond symbol is determined to include all tangent constraints of thetangent pixel.
 3. The symbol matching method of claim 1, wherein theaccepting of the match is responsive to a determination that the secondsymbol includes for each of the at least a subset of the tangent pixelsof the first symbol a corresponding pixel that includes one or more sametangent constraints.
 4. The symbol matching method of claim 1, whereinthe subset includes only tangent pixels neighbored by another tangentpixel that has at least one same tangent constraint as that of thetangent pixel.
 5. The symbol matching method of claim 1, wherein pixelsof the first and second symbols correspond if they are not offset fromeach other by more than one with respect to their respective bitmaps. 6.The symbol matching method of claim 5, wherein the acceptance of thefirst and second symbols as a match based on shared tangent constraintsis condition upon that, for each of the at least the subset of thetangent pixels, the corresponding pixel that includes the at least onesame tangent constraint as that of the tangent pixel is not offset fromthe tangent pixel in a direction that does not parallel a tangentdirection of the tangent pixel.
 7. The symbol matching method of claim1, wherein a number of shared constraints required for acceptance of thefirst and second symbols as a match based on the shared constraintsdepends on a height, width, and a mean stroke thickness.
 8. The symbolmatching method of claim 1, further comprising: for each of the at leasta subset of the tangent pixels: analyzing eight pixel neighbors of thetangent pixel; and determining, base on the analysis of the eight pixelneighbors, a number of the at least one same tangent constraint requiredfor use of the constraint similarity between the tangent pixel and itscorresponding pixel as a basis for accepting the first and secondsymbols as a match.
 9. The symbol matching method of claim 1, furthercomprising: for each of the at least a subset of the tangent pixels:determining whether the first symbol includes another pixel that issymmetric to the tangent pixel with respect to an axis; and determining,based on the determination regarding the inclusion of the symmetricpixel, a number of the at least one same tangent constraint required foruse of the constraint similarity between the tangent pixel and itscorresponding pixel as a basis for accepting the first and secondsymbols as a match.
 10. The symbol matching method of claim 1, whereinthe symbol matching method is used for data compression.